Accurate floating point summation∗

We present and analyze several simple algorithms for accurately summing n floating point numbers S = ∑n i=1 si, independent of how much cancellation occurs in the sum. Let f be the number of significant bits in the si. We assume a register is available with F > f significant bits. Then assuming that (1) n ≤ b2F−f/(1 − 2−f )c + 1, (2) rounding is to nearest, (3) no overflow occurs, and (4) all underflow is gradual, then simply summing the si in decreasing order of magnitude yields S rounded to within just over 1.5 units in its last place. If S = 0, then it is computed exactly. If we increase n slightly to b2F−f/(1− 2−f )c+ 3 then all accuracy can be lost. This result extends work of Priest and others who considered double precision only (F ≥ 2f). We apply this result to the floating point formats in the (proposed revision of the) IEEE floating point standard. For example, a dot product of IEEE single precision vectors ∑n i=1 xi ·yi computed using double precision and sorting is guaranteed correct to nearly 1.5 ulps as long as n ≤ 33. If double extended is used n can be as large as 65537. We also show how sorting may be mostly avoided while retaining accuracy. ∗Computer Science Division Technical Report UCB//CSD-02-1180, University of California, Berkeley, 94720. This research was supported in part by LLNL Memorandum Agreement No. B504962 under the Department of Energy Contract No. W-7405-ENG-48 and DOE Grant No. DE-FG03-94ER25219, the National Science Foundation under Grant No. ASC-9813362, NSF Cooperative Agreement No. ACI-9619020, NSF Infrastructure Grant No. EIA-9802069, the National Science Foundation Graduate Research Fellowship, and by a gift from Intel. The information presented here does not necessarily reflect the position or the policy of the Government and no official endorsement should be inferred. †Computer Science Division and Mathematics Department, University of California, Berkeley, CA 94720 ([email protected]). ‡Computer Science Division, University of California, Berkeley, CA 94720 ([email protected]).